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ABSTRACT 




A new discrete reliability growth model is created by 
modifying the often used Army Material Systems Analysis 
Activity (AMSAA) continuous reliability growth model. The new 
model is labeled the AMSAA-D model. Its accuracy is evaluated 
and compared with three other existing discrete reliability 
growth models. The results show the AMSAA-D model is at least 
as accurate as the other models. In particular it is more 
accurate than an AMSAA discrete model which requires computer 
supported numerical methods to calculate the reliability 
estimates from test data. The AMSAA-D reliability estimates 
can be made with a hand-held calculator. Computer simulations 
were used to generate test data needed for the evaluation. The 
simulated test plan assumes that repeated tests on a system 
are performed until a predetermined number of failures occur, 
at which time a design change is made to the system to improve 
its reliability. Ten reliability values are specified to 
define a reliability growth pattern. Five hundred replications 
of each growth pattern are simulated. For each replication, 
reliability estimates are calculated for each of the ten sets 
of generated test data using equations from each of the four 
growth models. Averages and sample mean square error values 
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across the 500 replications are used to determine accuracy. 
Sensitivity of the AMSAA-D model to the number of failures 
before system modification and to the number of possible 
failure causes is also evaluated. Results of all evaluations 
are presented graphically. 
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THESIS DISCLAIMER 



The reader is cautioned that computer programs developed 
in this research may not have been exercised for all cases of 
interest. While every effort has been made, within the time 
available, to ensure that the programs are free of 
computational and logic errors, they cannot be considered 
validated. Any application of these programs without 
additional verification is at the risk of the user. 
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I . INTRODUCTION 



Early in the acquisition process of a system, a variety 
of test programs are conducted to detect weaknesses in its 
design features and manufacturing processes. These tests are 
performed under tight budget constraints; consequently the 
number of component, assembly and system tests is severely 
limited. These development testing programs are designed to 
induce early failures and determine associated failure causes. 
This information is needed to make appropriate changes in the 

design and production processes. Hopefully these changes will 

( 

yield an increase in the reliability of the system under 
acquisition. Current DoD policy requires reliability growth 
management and assessment programs for major systems 
acquisitions . 

Reliability growth methodology used to estimate the actual 
reliability growth patterns should be able to track a variety 
of growth patterns reasonably well. Many reliability growth 
models have been developed and reported in the literature. 
New ones are continually being developed and proposed for use 
by contractors. 

A discrete version of the well known Army Material System 
Analysis Activity ( AMSAA ) or DUANE model is analyzed in this 
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thesis. This discrete model uses the number of observed tests 



to achieve a prescribed number of failures as the primary data 
input. This model is evaluated and compared with other 
established models which have been extensively tested and 
described in former theses [Ref. 1,2,3]. The same Monte Carlo 
simulation is used as in the former theses to make the 
comparisons. The simulation was originally developed by James 
E. Drake [Ref. 1], then twice updated and enlarged in its 
performance and ability to track different reliability growth 
patterns by James D. Chandler, Jr. [Ref. 2] and Pam Markiewicz 
[Ref. 3]. The program and its features are well described in 
these theses and will not be included here. Only the program 
introduced to estimate the new model is presented. 

In Chapter II the analysis of the proposed model is 
developed, the simulation data presented and the results 
evaluated. A summary and recommendations for further studies 
is presented in Chapter III. 
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II. RELIABILITY GROWTH MODELING 



A proposed reliability growth model should be able to 
track accurately all likely reliability growth patterns. 
During the development phase of a system, the actual pattern 
of reliability growth is dependent on many factors. 
Reliability progress paths may be steadily increasing or they 
may indicate early stagnation and degradation prior to 
continued growth. They may also demonstrate no reliability 
progress at all [Ref. 4]. Figure 1 shows the true reliability 
growth patterns which will be used for evaluation purposes in 
this thesis. The different patterns are numbered from 1 to 6 . 
Pattern numbers cited later in this thesis refer to Figure 1. 

Reliability growth usually occurs in discrete jumps; 
therefore it is meaningful to break the system tests into 
subphases. Tests within the same phase have the same 
probability of success, i.e., the same reliability. 
Reliability may change from phase to phase. Hopefully it is 
increasing as the items under test undergo change from phase 
to phase. 



3 



RELIABILITY QROUTH PATTERNS 




Figure 1 : Possible reliability growth patterns 

A- THE DISCRETE AMSAA MODEL 
1. Theory 

The Discrete AMSAA model was developed in 1983 by L. 
Crow [Ref. 5] It stems from a cumulative continuous model of 
the foirm 

K(t) = Xt^ 

where K(t) denotes total number of failures after t hours of 
testing . 
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In the discrete model 



Ki = 2 M. 

3=1 J 

where M. denotes the number of failures in test phase j. 
^ i 

T. = 2 N. 

where Nj denotes the number of tests in test phase j . 

The model then states 



E(K^) = E(M3^) = (1 - 

E(K2> = E(M^ + M 2 ) = XT^+ (1 - R2^^2 ^ ^'^2 
Thus (1 - R 2 >N 2 = AT^ - AT^ 



Continuing in this manner the AMSAA discrete model states 



(1 



Ri)Ni 




AT 



)8 

i-1 



Maximum likelihood estimates for A and ^ are used to 
obtain the MLE for R^. The estimates are updated sequentially 
as the test results of the next test phase become available. 

2 . Some Results 

The results shown in the following figures are taken 
directly from a thesis by Major Rio M.Thalieb [Ref. 6]. 

Figure 2 shows the estimation of the reliability 
growth when the actual reliability is following pattern 1. 

Figures 16 and 17 in Appendix D show the estimation 
of the discrete AMSAA model for patterns 2 and 4 . 
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RCLIAHILITY GROWTH ESTIMATION 




e 2 4 e fi 10 

Figure 2 : AMSAA reliability growth estimation, pattern 1 

It is clear that the discrete AMSAA model is not able 
to track all possible reliability growth patterns, especially 
when the growth is not steadily concave increasing. This is 
an expected result due to the fact that the results of all 
phases are combined to estimate the reliability. 

B. THE AMSAA-D MODEL 

The model analyzed in this thesis is a discrete variation 
of the continuous AMSAA model for estimating the failure rate 
of a system. The input random variables to the continuous 
model are the total number of failures and the total 
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accumulated test time up to the point of estimation [Ref. 4]. 
It is different from the discrete AMSAA model developed by 
Duane and evaluated in the thesis of R.M. Thalieb. [Ref. 6] . 
The equation used by AMSAA to model the instantaneous failure 
rate f^^ is 



f^T = b(l - a)TT"^ ( II. 1 ) 

where TT denotes the total test time and a and b are 
parameters. The concept for this model was first presented by 
Duane [Ref. 7]. A large amount of development test data he 
examined indicated that the logarithm of the failure rate was 
a linear function of the logarithm of the total accumulated 
test time. Comments on the development of this model appear 
in Appendix E. Woods [Ref. 4] showed that a modification to 
this model, using regression estimation methods rather than 
maximum likelihood estimation methods, provided a more 
accurate growth model. The new discrete model developed here 
uses similar regression estimation methods. 

1. Theoretical analysis 

The discrete modification of the continuous AMSAA 
model developed here should be. distinguished from the Discrete 
AMSAA model and will hereafter be referred to as the AMSAA-D 
model. We shall adopt the following notation : 

j or k : test phase index, corresponds to design change 
index 
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f j : actual failure rate during phase j . Equal to 
probability of failure for a test, 
f^j^ : failure rate at the end of phase k as determined by 
the model . 

Fj : total number of failures observed in phase j 
Nj : number of trials in phase j 

Ntk • total accumulated number of trials up to and 
including phase k 

( Ntk = Ni + N2 + • • • + Nk ) 

Cj : number of possible failure causes in 
phase j 

We shall model the failure rate ft}- after a total 
number of tests Ntk have been accumulated over k phases by 

ftk = b(l - a)Ntk ( II-2 ) 



This provides a discrete version of the model given 
by equation (II. 1). Regression methods similar to those used 
by Woods [Ref. 4], will be used to estimate the parameters a 
and b chronologically as test phases are completed. That is, 
all observed numbers of failures F^, F 2 , ... F}. and the 
observed total numbers of trials N^^, N 2 , ... N}- are used to 
obtain the current estimate of f^j^ , k=l,2 .... 

Each test phase is terminated after a predetermined 
number of failures has occurred. Consequently the number of 
failures per phase will always be greater than 0. The maximum 
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likelihood estimate for fj is F^/Nj . Its expected value is 
larger than f j , the probability of failure on each trial. 
There are numerous ways to decrease the bias. The reduction 
factor chosen here is 2Cj/(l+2Cj) primarily because it 
resembles the bias reduction factor used by Woods [Ref. 4]. 
Therefore a more nearly unbiased estimate for fj is used 
where 






2C- 



1 + 2Cj Nj 



if Cj > 3 



0.85 



N- 



if Cj = 1,2 



(II. 3) 



A rationale for equation (II. 3) can be developed from 
a continuous analog. If = min(T^,tQ), i = 1,2, . . . ,N where 

Tj^, ..., Tj^ are iid exponential variables, then a nearly 

A 

unbiased estimate A for the common failure rate A is given by 



2N F 
1+2N 13^ 



where F is the number of failures (T^ < t^) . This bias 
correction factor, 2N/(1+2N) is developed in Reference 8. 
Woods [Ref. 4] uses this correction factor for the continuous 
model. It is modified here by replacing Nj with C j . Table I 
shows a comparison of both correction factors 2Cj/(l+2Cj) and 
2Nj/(l+2Nj). The chosen correction factor in equation (II. 3) 
is a constant when the number of failure causes or the number 



9 



of subsystems in each phase are fixed. Table I shows both the 
smallest and the largest values of the correction factor 
2Nj/(l+2Nj) across all 500 replications and six growth 
patterns for j = 1 and j = 10, i.e. for phases 1 and 10. These 
four values are grouped by number of failure causes. The 
number of failure causes is an input parameter for the 
simulation program. The number of failure causes is important 
when a fraction of a failure for a specific cause can be 
removed after a fixed number of successful! system tests 
occur without failure for the same cause. This failure 
reduction process is called failure discounting and is 
permitted by specifying input parameters to the computer 
program used to run these simulations. Failure discounting was 
not allowed in any simulation in this thesis. 



TABLE I : COMPARISON OF CORRECTION FACTORS 







# of failure 


causes Cj 








1 


3 


5 


7 


AMSAA-D 2C^/(1+2C^) 


0.85 


0.8571 


0.9091 


0.9333 


continuous 

AMSAA 

model 

2Nj/(l+2Nj) 


min 

phase 1 

max 


0.8571 

0.9524 


0.8571 

0.9565 


0.8571 

0.9565 


0.8571 

0.9333 


min 

phase 10 

max 


0.9524 

0.9915 


0.9091 

0.996 


0.96 

0.9948 


0.9231 

0.9922 



Least square estimation procedures are used to obtain 
estimates for a and b as the testing in each new phase is 
completed. This provides update estimates for a and b and thus 
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for the failure rate f ^ j ' ^ .... Equations for the least 
square estimates were first developed by Neal [Ref. 9]. He 
uses the transformations 

Xj = In N^j and 
Yj = In f^j 

to obtain a linear relationship between Yj and X j . His 
equations for the least squares estimates aj^ and are as 
follows : 



A 





k 

2 X. 



j=i 




X 



j 




X 



2 

j 






1 - a. 






Sk^'k 



where 



Xv = - 2 X-i 

^ k ^ 



Yk 




Y 



j 



k > 2 . 

The regression estimation can be made only after two 
phases of testing have been completed. Therefore the failure 
rate is estimated for k > 2 as 






tk 



( 1 - a^ ) 6v N 



tk 



( II. 4 ) 



The term f^^ = f^ is estimated with equation ( II. 3 ). 
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2 . Reliability estimate as a linear function of the 
failure rate estimate. 

To compare the results of this model with other 
models, it is necessary to convert the failure rate estimate 
into a reliability value. 

A discrete failure rate is defined as 



^ P[N=n] 
^ P[N>n] 



n = 1,2,3... 



( II-5 ) 



where the random variable N is the number of cycles (trials) 
up to and including the cycle on which the r^^ failure occurs 
[Ref. 10]. For the geometric distribution r = 1 and f^^ = 1-p. 
When estimating system reliability in each phase, one basic 
assumption is that the unknown reliability, p, stays constant 
within that phase. Even though we are observing the number of 
trials to the r^^ failure with common probability of success 
p on each trial, we shall take the failure rate to be the 
unreliability 1-p. This means that the failure rate is just 
the probability of failure on a single trial inside one single 
phase. Consequently one method for converting the least 
squares failure rate estimates into reliability estimates is 
from the expression : 

Rtk = 1 - ftk ( ) 



where f^j, 
subscripts 



is 

in 



given in equation (II. 4). In order to avoid 
graphs, equation (II. 6) will be written as 
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R(N) = 1 - F(N) in the graphs. In the text that follows it is 
referred to as "the linear model". A more detailed discussion 
of the relationship between failure rate and reliability for 
discrete distributions is provided in Appendix C. 

3. Reliability estimate as an exponential function of the 
failure rate estimate. 

Analysis of the first simulations indicated that the 
linear model consistently underestimated the actual system 
reliability on the average. To increase the value of the 
average point estimate, another equation was introduced for 
estimating reliability using the failure rate. This equation 
is 



Rtk 




( ii-v ) 



and is referred to as " the exponential power model " in the 
text. In the figures it is noted as 



R(N) = exp(f (N) ) . 

4 . Simulation results 

The simulations were replicated 500 times for each 
combination of input parameters. These parameters are 

- the phase reliabilities p^ ... P^q* Six different sets of 
these 10 values establish six different reliability growth 
patterns , 

- the number of failure causes per phase 
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the number of failures before corrective action in each 



phase. 

Table II gives an overview of all simulated parameter 
combinations . 



Table II : ALL SIMULATED PARAMETER COMBINATIONS 



# of 


# of 


pattern 


causes 


fail/phase 


1 


2 


3 


4 


5 


6 


3 


3 


X 


X 


X 


X 


X 


X 




1 


X 


X 


X 


X 


X 


X 




2 


X 


X 


X 


X 


X 


X 


5 


3 


X 


X 


X 


X 


X 


X 




5 


X 


X 


X 


X 


X 


X 




7 


X 


X 


X 


X 


X 


X 


7 


3 


X 


X 


X 


X 


X 


X 



During 
reliability as 
computed. Err^ 



each replication the 
well as the estimation 
is defined as 



Err . 
J 



R . 



D 



R 



tj 



where 



estimated system 
error, Err j , was 

( II. 8 ) 



Rj : actual system reliability in phase j 

A, 

R^j : estimated system reliability at the end of 
phase j 

These computations are performed for both AMSAA-D 
models ( the linear and the exponential power model ) . They 
are also computed for the Maximum Likelihood Estimation model 
and the exponential Regression Estimate model; hereafter 
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referred to as the MLEwd and Exp . Reg . Estimate models. These 
models have been evaluated by Chandler [Ref. 2]. Additional 
comments on these models are given in Appendix E. After all 
replications are completed the average system reliability 
estimate and the Mean Square Error are computed for each 
estimation method as 



nrep 



R^^ = 



tj nrep 



2 (R.) 



tj 



for all j ( II. 9a ) 



nrep _ 

MSE . = 2 [Err.] for all j ( II.9b ) 

3 nrep '• i-' ’ 



where j 

nrep 

"7T 

R 



tj 



MSE 



: phase index 

: total number of replications ( = 500 ) 

: average reliability estimate over nrep 
replications for phase j 
: Mean Square Error 
This average reliability estimate and the Mean 
Square Error are plotted and graphed to show the performance 
of the growth models. 

The MLEwd and the exp . Reg. Estimate models are used to 
compare the performance of both AMSAA-D models relative to 
these former established growth models. The AMSAA-D model does 
not allow the application of any discounting method. Therefore 
the comparisons are all made without any discounting applied 
to the MLEwd and the exp. reg. Estimate. 
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a. Sensitivity of the reliability growth estimation with 
respect to different growth patterns 
The evaluation of the sensitivity of both 
AMSAA-D growth models with respect to different growth 
patterns is discussed with the following parameter setting : 

- number of failure causes : 5 

- number of failures/phase : 3 

The results and conclusions, however, are in general the same 
for all other parameter combinations as is shown in Figures 
18 - 35 in Appendix D. 

Figure 3 illustrates the performance of both 
AMSAA-D models ( linear and exponential ) when the true 
reliability growth follows the path stipulated in pattern 1. 

The exponential power model in the first 6 phases 
far overestimates the actual reliability. The linear model 
tracks the convex growth pattern very well over all phases. 
Also it underestimates the actual reliability, which is 
usually a desired behavior. In comparison with the MLEwd and 
the exp. Reg. Estimate, the linear model performs better over 
all phases with respect to the average of the point estimates 

A A 

^tl' ^t2 ' ‘ ■ 

The Mean Square Error for all four methods is 
shown in Figure 3. The exponential power model is the worst 
estimation model with respect to the average reliability 
estimate, but it has the smallest Mean Square Error and is 
less dependent on the phase. But from phase 8 on, all models 
have nearly the same Mean Square Error. 
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Figure 3 : Reliability growth estimation, pattern 1 
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Figure 4 illustrates the performance of all models 
under pattern 2 . 

Both AMSAA-D models track the decline of the 
actual reliability very well. The linear model again always 
underestimates the actual system reliability, and the 
exponential power model overestimates the actual system 
reliability up to phase 6. It also smoothes the decline and 
the following incline of the actual reliability. 

The linear model performs better with respect to 
the average reliability estimate, when compared with the MLEwd 
and the exp . Reg . Estimate . Also, from phase 3 on, both AMSAA- 
D models have the smaller Mean Square Error and this Mean 
Square Error is not very strongly influenced by the phase as 
is the case for the MLEwd and the exp. Reg. Estimate. 

For pattern 3 the general behavior of all models, 
as shown in Figure 5, is similar to the behavior for patterns 
1 and 2 . 

All models perform well under pattern 4 and 5 as 
it can be seen in Figures 6 and 7 . For pattern 5 only the 
exp . reg . Estimate shows an overestimation of the actual 
reliability over all phases. Again, the decline of the Mean 
Square Error from phase 1 to phase 3 for all models is very 
significant. 

Figure 8 shows the behavior under pattern 6. All 
models underestimate the actual reliability quite well in the 
average. The Mean Square Error is nearly the same for all 
models and is also nearly constant over all phases. Only the 
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Figure 4 : Reliability growth estimation, pattern 2 
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Figure 6 : Reliability growth estimation, pattern 4 
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Figure 8 : Reliability growth estimation, pattern 6 
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exp. Reg. Estimate shows a slight decline in the Mean Square 
Error from phase 1 to 10. 

Figures 9 and 10 illustrate the behavior of both 
AMSAA-D models with respect to the Mean Square Error and with 
phases and patterns as variables. 




Both models show a high Mean Square Error in the 
early phases. This is reasonable because in the early phases 
less information is available to get accurate reliability 
estimates. The exponential power model ( Figure 10 ) shows a 
constant decline of the Mean Square Error for pattern 1, 
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function of phases and patterns 

whereas for the pattern 2-5 the Mean Square error drops 
rapidly after phase 1 and stays nearly constant thereafter. 

For both models pattern 6 is less dramatic because 
both models show a nearly constant low Mean Square Error over 
all phases. 

b. Sensitivity of the reliability growth estimation to the 
number of failure causes. 

To evaluate the sensitivity of the reliability 
growth estimation to the number of failure causes the 
simulation was run for each pattern with the number of failure 
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causes set at 3 , 5 and 7. Figure 11 and the Figures 36 to 40 
in Appendix D show three dimensional plots of the Mean Square 
Error as a function of the phase and the number of failure 
causes for the linear and the exponential power model. The 
Mean Square Error was chosen as a measure of the sensitivity, 
because the estimation of the actual reliability doesn't show 
great differences due to the change in the number of failure 
causes. The graphs of the reliability estimation and their 
related Mean Square Error for all different parameter settings 
is enclosed in Appendix D ( Figures 18 - 35 ) . 

As can be seen, the general behavior of the growth 
estimation due to the number of failure causes is relatively 
independent of the simulated growth pattern. Also the 
difference between the two models is not very large. Both 
models show, in general, an increasing Mean Square Error when 
the number of failure causes is increased. This increase is 
overall a little steeper for the linear model, but it becomes 
less dependent in the number of causes in later phases. 

The exponential power model shows a more 
differentiated behavior. It has a steep increase of the Mean 
Square Error for pattern 1 in the first 5 phases as the number 
of causes is increased. This is true in all other patterns 
only for phase 1. For all other patterns and phases greater 
than 1 the Mean Square Error of the exponential power model 
is only slightly increasing with increasing number of causes. 
The slope is reduced even more within the last phases. 



26 



mean SQUAPE EPPOR t PATTERN 1 




Figure 11: Mean Square Error as a function of phases and 

failure causes, both AMSAA-D models, pattern 1 
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To catch the general trend in a different way a 



total average Mean Square Error was computed as 



tot.avg.MSE = 



npat 



"pat p_i 



- nphase 

— ^ — 2 MSE . ) 

nphase jp ' 



(II. 10) 



where 



HSE.p 



: Mean Square Error as in equation (II. 9), 
j : phase index ( here 1 to 10 ) , 

p : pattern index ( here 1 to 6 ) , 

nphase : number of phases ( here 10 ) and 

npat : number of patterns ( here 6 ) . 

The result is shown in Figure 12. It also affirms 
the trend estimation from the three dimensional graphs. The 
total average Mean Square Error increases for both models. 
This is also true for the other two models, MLEwd and 
exp . Regr. Estimate , shown for comparison. But the slope of the 
linear model is steeper than the slope of the exponential 
power model when increasing the number of causes from 3 to 5 . 

c. Sensitivity of the reliability growth estimation to the 
number of failures per phase 

The evaluation of the sensitivity of the 
reliability growth estimation to the number of failures per 
phase was done with the following parameter settings : 

- all patterns ( 1 to 6 ) 

- number of failure causes = 5 
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Figure 12 : Average Mean Square Error as a function of failure 
causes 



Figure 13 and Figures 41 to 45 in Appendix D show 
the three dimensional plots of the variation of the Mean 
Square Error over all phases with varied failures/phase for 
all patterns. 

From all Figures, it can clearly be seen that the 
greatest impact on the reduction of the Mean Square Error 
occurs where the number of failures/phase changes from 1 to 
2. In general an increase in the number of failures/phase from 
1 to 2 will reduce the mean Square Error by at least 50% in 
most of the cases. This is relatively independent of the 
phase. But the reduction in phase 1 is always the smallest for 
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Figure 13 : Mean Square Error as a function of phases and 

failures per phase, both AMSAA-D models, pattern 1 
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each pattern. Note that there is no decrease of the Mean 
Square Error for the exponential power model in the first 
phase for the patterns 1 to 5 when the number of 
failures/phase is increased. Only pattern 6 shows a decrease 
in phase 1 also. 

The variation of the total average Mean Square 
Error over all phases and patterns, computed as in equation 
II. 10, with increasing number of failures/phase is shown in 
Figure 14. It again affirms in general the trend evaluated 
from the three dimensional graphs. The greatest reduction of 
the Mean Square error is achieved by increasing the number of 
failures/phase from 1 to 2. This results in an average 
decrease of the Mean Square Error from 0.08 to 0.03 for the 
linear model and from 0.03 to about 0.015 for the exponential 
power model. This result is true in general for the Maximum 
Likelihood Estimation and the exponential Regression Estimate, 
which are also shown in Figures 13 and 14 for comparison. 
d. Comparison with the discrete AMSAA model 

The results show the AMSAA-D model is more 
accurate than the discrete AMSAA model. The AMSAA-D model 
appears to account more accurately for different reliabilities 
in different phases, but still uses information from previous 
phases to yield more accurate estimates of reliability at 
later stages in the growth process. It makes good use of the 
fact that the actual reliability is not constant over all 
phases, but stays constant within a phase. The discrete AMSAA 
model is not able to track all different possible patterns of 



31 



MCfW SQUARE E RROR I AVERAGE OP ALL PATTERN AND PHASES 




Figure 14 : Average Mean Square Error as a function of 
failures per phase 

reliability growth as can be seen in Figure 15, which is taken 
from R.Thalieb [Ref. 6] for pattern 2 with 3 failures/phase 
and 1 failure cause. 

e. Conclusion 

The linear AMSAA-D model performs reasonably well 
and is relatively independent of the pattern of actual 
reliability growth. It always underestimates the actual 
reliability and is therefore a conservative approach. It also 
performs at least as good as the formerly established MLEwd 
and exp . Regr . Estimate models when using Mean Square Error as 
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RELIABILITY GROWTH ESTIMATION 



diacrata AMSAA and AMSAA-D modal 




Figure 15 : Comparison of the discrete AMSAA model with the 
AMSAA-D model, pattern 2 

a measure of performance. It is not very sensitive to the 
number of failure causes and this low sensitivity is even 
decreasing within later phases of testing. When the number of 
failures/phase are increased from 1 to 2 , the Mean Square 
Error can be reduced by about 50%. 

The exponential power model has a very low Mean 
Square Error. In general, it is about 50% less than the Mean 
Square Error of the other models. It overestimates the actual 
reliability in the early phases for nearly all of the six 
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growth patterns . The impact on accuracy due to an increase in 
the number of failure causes is nearly the same as for the 
linear model. The same can be said for the impact of an 
increase in the number of failures/phase. 

The linear AMSAA-D model is a conservative, but 
reasonable accurate tool, for estimating actual system 
reliability growth. It is not very sensitive to the number of 
failure causes and an increase in the number of failures per 
phase can reduce the Mean Square Error significantly. 
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III. SUMMARY, RECOMMENDATIONS 



A. SUMMARY 

The objective of this thesis was to evaluate the accuracy 
of a discrete modification of the continuous AMSAA model 
( AMSAA-D model ) for the reliability growth estimation. The 
sensitivity of the model to different growth patterns and 
system parameters as the number of subsystems, the number of 
phases and the number of failures till phase termination was 
evaluated. The accuracy of this model was compared with that 
for the MLEwd and the exponential regression estimate models 
[Ref. 1,2,33. Although possible improvements due to different 
discounting and weighing methods for the MLEWD and the 
exponential regression estimate models have been demonstrated 
in these former theses, comparisons in this thesis were made 
without using discounting or weighting of data. 

Throughout the simulations the AMSAA-D model illustrated 
a reasonable accuracy and was at least as good as the MLEWD 
or the exponential regression estimate. It is able to track 
all possible growth patterns reasonably well. Its sensitivity 
to the number of failure causes and the number of failures 
within a phase compares favorably with that of other 
established models. 
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B. RECOMMENDATIONS 



The following are recommendations for further study : 

1. The AMSAA-D model should be modified by using the 
minimum variance unbiased estimate of the failure rate 
instead of the failure rate estimate using a 
correction factor. 

2. Since the exponential regression estimate can be 
improved by weighting the data, similar methods should 
be analyzed for the AMSAA-D model. 
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APPENDIX A 



USER'S GUIDE TO : RELIABILITY GROWTH WITH THE AMSAA-D MODEL 

( DISCONT ) 



1. Introduction 

2. The DISCONT EXEC A1 file 

3. The input data file 

4. The output file GROWTH EST A1 
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APPENDIX A : USER'S GUIDE TO 
RELIABILITY GROWTH 
WITH THE 
AMSAA-D MODEL (DISCONT) 



1 . Introduction 

In order to use the Fortran program DISCONT the user 
must possess three files : 

- Input file "like" PT1C5F3 DATA A1 

- DISCONT FORTRAN A1 

- DISCONT EXEC A1 

A sample of each of these files along with a sample 
run using DISCONT EXEC A1 and a sample output is contained in 
Appendix B. The input file and the exec file can be changed 
to meet the needs of the user. In the current form, the exec 
file produces a large number of intermediate calculations for 
both the DISCONT FORTRAN program and Lt P. Markiewicz's 
program DRG FORTRAN [Ref. 3]. These calculations may not be 
of interest and therefore may be eliminated with no effect to 
the simulation. A detailed explanation of each file, which is 
created in addition to Lt P.Markiewicz ' s version is contained 
in the following sections. 

2. The DISCONT EXEC A1 file 

The DISCONT EXEC A1 contains all necessary file 
definitions and commands needed to run DISCONT FORTRAN Al. 
Appendix B 2. shows a sample run using DISCONT EXEC Al. 
DISCONT FORTRAN Al is an extension of the DRG FORTRAN Al 
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program of Lt P.Markiewicz [Ref. 3]. Therefore in addition to 
the output created for the discretized continuous models there 
are all outputs and models available, which have been used by 
Lt P.Markiewicz [Ref. 3]. 

For the use of DISCONT FORTRAN A1 it is necessary to 
specify an Input file in line 24 of DISCONT EXEC A1 . This 
input file must have all necessary inputs in the same format 
as mentioned in Appendix A of Lt P.Markiewicz thesis [Ref. 3]. 
The filename and type can be chosen free as long as it matches 
the format of the CP/CMS requirements of a maximum of eight 
characters. 

For the purpose of this thesis, 42 different input 
files have been created to allow the evaluation of the 
sensitivity of the model to the following different input 
parameters 

- type of growth pattern 

- number of failure causes and 

- number of failures per phase. 

The filenames, used in the 42 input files, express the 
setting of these different input parameters. For example, the 
filename PT1C5F3 means 

- pattern : i 

- number of failure causes : 5 

- number of failures/phase : 3 

The full name of the input file is then 
PT1C5F3 DATA A1 . 
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The output for the AMSAA-D models as well as the 
results of the models chosen for comparison is written to 
GROWTH EST A1 . A sample output is given in Appendix B. 

Also a control output is available. It allows us to 
track the intermediate results of single simulation runs for 
the first and last 10 replications, independent of the number 
of replications chosen. This control output is written to 
GROWTH CONTROL A1 . 

The next output file is FAILURE RATE Al. It contains 
the actual reliability as well as the average of the estimated 
failure rates of each replication, which is the basis for the 
computations of the estimated reliability in the linear and 
the exponential power model. 

The last output file is MSE COMPARE Al . This file 
contains results used to evaluate the influence of changes in 
the number of failure causes and the number of failures/phase. 
It contains the average Mean Square Error, averaged over all 
phases from the distinct Mean Square Error for each phase, 
computed for the four growth models 

- 1 .... linear model 

- 2 .... exponential power model 

- 3 .... MLE with discounting 

- 4 .... exponential Regression Estimate 

In addition to these files, related to the AMSAA-D 
models, all results of Lt P.Markiewicz ' s simulation program 
DRG FORTRAN Al are computed and written to the same output 
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files as in their original program. The only difference is 
that the filenames have in addition an "X" in front of their 
original name. In this way it is possible to run both 
simulations, DISCONT and DRG, one after the other without 
overwriting the first created output files. Therefore the file 
descriptions in Appendix A of the thesis of Lt P.Markiewicz 
are totally valid. 

3. The input data file 

These instructions should be used in connection with 
a sample input ( see Appendix B 2.). The sequence of all data 
inputs is mandatory as it is shown in the sample input file. 
The input is read from file device number 10. 

For the discretized continuous models the input can 
be the same as for the DRG FORTRAN program of Lt. 
P.Markiewicz. All steps noted in her thesis in Appendix A 3. 
are still valid. The only difference is that the first line 
of the input file should contain the filename of the input 
file to get this remark stated on top of the output file 
GROWTH EST A1 . Due to this additional first line the only 
change in the steps of Appendix A 3 of Ref. 3 is that the 
mentioned line numbers must be increased by one. But all other 
input variations/possibilities stay the same. 
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4. Output file GROWTH EST A1 

The output file shows in its first line the filename 
of the input file used for the actual simulation run. The 
names of the variables are : 



- PHASE 


, self explanatory 


- ACTREL . . , 


. actual reliability 


- LINREL . . , 


, reliability estimate, using the linear model 


- MSEL 


. Mean Square Error of the linear model 


- RELEXP . . . 


reliability estimate using the exponential 
power model 


- MSEE 


. Mean Square Error of the exponential power 
model 


- MLEWD 


. Maximum Likelihood estimate with discounting 


- MSEWD 


, Mean Square Error of the MLEWD 


- REGEST . . , 


. exponential regression estimate 


- MSEREG . . . 


. Mean Square Error of the exp. Reg. Estimate 
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APPENDIX B 



FILES AND VARIABLES 



1. Sample DISCONT EXEC A1 file 

2. Sample input file : PT1C5F3 DATA A1 

3. Sample simulation run, using DISCONT EXEC A1 , 
recorded session 

4. Sample output GROWTH EST A1 

5. Main variables used for the discrete modification 
of the continuous AMSAA model ( AMSAA-D model ) 
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1. Sample DISCONT EXEC A1 file 



&TRACE OFF 

******************************************************* 
&FN = DISCONT 
&FN1 = GROWTH 

&TYPE THIS PROGRAM USES A DISCRETIZED CONTINUOUS MODEL 
&TYPE DO YOU NEED TO COMPILE YOUR PROGRAM ? (Y/N) 

&READ VAR &R_COMPILE 

&IF &R_COMPILE NE Y &GOTO -RUN 

-H FORTVS &FN 

&IF &RC EQ 0 &GOTO -RUN 

&TYPE Your program did not compile; check for errors. 
&TYPE DO YOU WISH TO VIEW THE PROGRAM LISTING FILE? (Y/N) 
&READ VAR &RSP1 

&IF &RSP1 EQ Y BROWSE &FN LISTING A 

&TYPE DO YOU WISH TO XEDIT THE PROGRAM FILE? (Y/N) 

&READ VAR &RESP1 

&IF &RESP1 NE Y &EXIT 1 

& COMMAND XEDIT &FN FORTRAN A 

&TYPE DO YOU WISH TO RUN THE PROGRAM AGAIN? (Y/N) 

&READ VAR &RESP2 

&IF &RESP2 EQ Y &GOTO -H 

&EXIT 1 

-RUN 
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FILEDEF 


10 


DISK 


PT2C7F3 DATA A1 






FILEDEF 


82 


DISK 


XAl NUM A1 






FILEDEF 


84 


DISK 


XA9 NUM A1 






FILEDEF 


30 


DISK 


XJRELIAB LISTING 


A1 (LRECL 


133 


FILEDEF 


35 


DISK 


XPRELIAB LISTING 


A1 (LRECL 


133 


FILEDEF 


20 


DISK 


XJTHESIS OUT A1 






FILEDEF 


25 


DISK 


XPTHESIS OUT A1 






FILEDEF 


81 


DISK 


XPMATRIXA LISTING 


(LRECL 133 


FILEDEF 


83 


DISK 


XJMATRIXA LISTING 


(LRECL 133 


FILEDEF 


87 


DISK 


XPREGMAT DATA A1 






FILEDEF 


88 


DISK 


XJREGMAT DATA A1 






FILEDEF 


90 


DISK 


XYSTAR LISTING (LRECL 133 




FILEDEF 


89 


DISK 


XTRIALS DATA A1 






FILEDEF 


40 


DISK 


XEST OUT A1 






FILEDEF 


50 


DISK 


XJMLEWD OUT A1 






FILEDEF 


55 


DISK 


XPMLEWD OUT A1 






FILEDEF 


60 


DISK 


XJMLESP OUT A1 






FILEDEF 


65 


DISK 


XPMLESP OUT A1 






FILEDEF 


70 


DISK 


XJREGEST OUT A1 






FILEDEF 


75 


DISK 


XPREGEST OUT A1 






FILEDEF 


15 


DISK 


XPWRESl OUT A1 






FILEDEF 


39 


DISK 


XPWRES2 OUT A1 






FILEDEF 


49 


DISK 


XPWRES3 OUT A1 






FILEDEF 


16 


DISK 


XJWRESl OUT A1 






FILEDEF 


38 


DISK 


XJWRES2 OUT A1 






FILEDEF 


48 


DISK 


XJWRES3 OUT A1 
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FILEDEF 


52 


DISK 


XMLEWDl NUM A1 


FILEDEF 


51 


DISK 


XMLEWDl SDV A1 


FILEDEF 


54 


DISK 


XMLEWD9 NUM A1 


FILEDEF 


53 


DISK 


XMLEWD9 SDV A1 


FILEDEF 


72 


DISK 


XREG8 


NUM 


A1 


FILEDEF 


71 


DISK 


XREG8 


SDV 


A1 


FILEDEF 


74 


DISK 


XREG16 


NUM A1 


FILEDEF 


73 


DISK 


XREG16 


SDV A1 


FILEDEF 


77 


DISK 


XMIPI 


NUM 


A1 


FILEDEF 


76 


DISK 


XMIPI 


SDV 


A1 


FILEDEF 


79 


DISK 


XM2P1 


NUM 


A1 


FILEDEF 


78 


DISK 


XM2P1 


SDV 


A1 


FILEDEF 


92 


DISK 


XM3P1 


NUM 


A1 


FILEDEF 


91 


DISK 


XM3P1 


SDV 


A1 


FILEDEF 


18 


DISK 


XM1P9 


NUM 


A1 


FILEDEF 


17 


DISK 


XM1P9 


SDV 


A1 


FILEDEF 


94 


DISK 


XM2P9 


NUM 


A1 


FILEDEF 


93 


DISK 


XM2P9 


SDV 


A1 


FILEDEF 


96 


DISK 


XM3P9 


NUM 


A1 


FILEDEF 


95 


DISK 


XM3P9 


SDV 


A1 


FILEDEF 


62 


DISK 


XMLESPl NUM A1 


FILEDEF 


61 


DISK 


XMLESPl SDV A1 


FILEDEF 


64 


DISK 


XMLESP9 NUM A1 


FILEDEF 


63 


DISK 


XMLESP9 SDV A1 


FILEDEF 


11 


DISK 


GROWTH 


[ CONTROL 


FILEDEF 


12 


DISK 


FAILURE RATE A1 
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FILEDEF 13 DISK GROWTH EST A1 



FILEDEF 14 DISK ERROR EST A1 

FILEDEF 19 DISK MSE COMPARE A1 

FILEDEF 21 DISK LLOYD EST A1 

FILEDEF 22 DISK FRATE COMPARE A1 

FILEDEF 06 TERMINAL 

LOAD &FN (START 

&IF &RC EQ 0 &SKIP 9 

&TYPE Your program did not run correctly; check for 
errors . 

&TYPE Do you wish to XEDIT the program file? (Y) 

&READ VAR &RESP3 

&IF &RESP3 NE Y &EXIT 2 

&COMMAND XEDIT &FN FORTRAN A 

&TYPE Do you wish to run the program again? (Y) 

&READ VAR &RESP4 

&IF &RESP4 EQ Y &GOTO -H 

&EXIT 2 

&TYPE LLOYD ESTIMATION/EXPECTATION OUTPUT IS IN "LLOYD EST 
Al" 

&TYPE 

&TYPE CONTROL OUTPUT IS IN THE FILE "GROWTH CONTROL Al" 
&TYPE 

&TYPE YOUR ESTIMATION OUTPUT IS IN THE FILE &FN1 EST Al 
&TYPE DO YOU WISH TO BROWSE YOUR ESTIMATION OUTPUT? (Y/N) 
&READ VAR &RESP 
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&IF &RESP EQ Y &COMMAND BROWSE &FN1 EST A1 
&TYPE PRINT YOUR OUTPUT FILE? (Y/N) 

&READ VAR &RESP7 

&IF &RESP7 EQ Y &COMMAND PRINT &FN1 EST A1 
-REDO 

&TYPE DO YOU WISH TO CHANGE THE INPUT DATA FILE (Y/N) 
&READ VAR &RESP5 

&IF &RESP5 EQ Y XEDIT &FN EXEC A1 

&TYPE DO YOU WISH TO RUN THE PROGRAM AGAIN? (Y/N) 
&READ VAR &RESP6 

&RESP56 = &CONCAT OF &RESP5 &RESP6 
&IF &RESP56 EQ YY &GOTO -H 
&IF &RESP6 EQ Y &GOTO -RUN 
&EXIT 

2. Sample input file : PT1C5F3 DATA A1 

PT1C5F3 PATTERN 1, 5 FAILURE CAUSES, 



3 FAILURES/ PHASE 



5 



NUMBER OF FAILURE CAUSES 



10 



NUMBER OF PHASES ( NPHASE ) 



1 



FIXED RELIABILITY OPTION 



( 1: YES ; 0: NO ) 



3 



NUMBER OF FAILURES IN PHASE 1 



3 



NUMBER OF FAILURES IN PHASE 2 



3 



NUMBER OF FAILURES IN PHASE 3 



3 



NUMBER OF FAILURES IN PRASE 4 



3 



NUMBER OF FAILURES IN PHASE 5 
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NUMBER OF FAILURES IN PHASE 6 
NUMBER OF FAILURES IN PHASE 7 
NUMBER OF FAILURES IN PHASE 8 
NUMBER OF FAILURES IN PHASE 9 
NUMBER OF FAILURES IN PHASE 10 



85 


PROB. 


OF 


SUCCESS 


FROM 


CAUSE 


1 


IN 


PHASE 


1 


86 


PROB. 


OF 


SUCCESS 


FROM 


CAUSE 


1 


IN 


PHASE 


2 


90 


PROB. 


OF 


SUCCESS 


FROM 


CAUSE 


1 


IN 


PHASE 


3 


91 


PROB. 


OF 


SUCCESS 


FROM 


CAUSE 


1 


IN 


PHASE 


4 


93 


PROB. 


OF 


SUCCESS 


FROM 


CAUSE 


1 


IN 


PHASE 


5 


95 


PROB. 


OF 


SUCCESS 


FROM 


CAUSE 


1 


IN 


PHASE 


6 


97 


PROB. 


OF 


SUCCESS 


FROM 


CAUSE 


1 


IN 


PHASE 


7 


99 


PROB. 


OF 


SUCCESS 


FROM 


CAUSE 


1 


IN 


PHASE 


8 


99 


PROB. 


OF 


SUCCESS 


FROM 


CAUSE 


1 


IN 


PHASE 


9 


998 


PROB. 


OF 


SUCCESS 


FROM 


CAUSE 


1 


IN 


PHASE 


10 


84 


PROB. 


OF 


SUCCESS 


FROM 


CAUSE 


2 


IN 


PHASE 


1 


85 


PROB. 


OF 


SUCCESS 


FROM 


CAUSE 


2 


IN 


PHASE 


2 


87 


PROB. 


OF 


SUCCESS 


FROM 


CAUSE 


2 


IN 


PHASE 


3 


90 


PROB. 


OF 


SUCCESS 


FROM 


CAUSE 


2 


IN 


PHASE 


4 


92 


PROB. 


OF 


SUCCESS 


FROM 


CAUSE 


2 


IN 


PHASE 


5 


95 


PROB. 


OF 


SUCCESS 


FROM 


CAUSE 


2 


IN 


PHASE 


6 


97 


PROB. 


OF 


SUCCESS 


FROM 


CAUSE 


2 


IN 


PHASE 


7 


99 


PROB. 


OF 


SUCCESS 


FROM 


CAUSE 


2 


IN 


PHASE 


8 


99 


PROB. 


OF 


SUCCESS 


FROM 


CAUSE 


2 


IN 


PHASE 


9 


998 


PROB. 


OF 


SUCCESS 


FROM 


CAUSE 


2 


IN 


PHASE 


10 


83 


PROB. 


OF 


SUCCESS 


FROM 


CAUSE 


3 


IN 


PHASE 


1 
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84 


PROB. OF 


SUCCESS 


FROM 


CAUSE 


3 


IN 


PHASE 


2 


86 


PROB. OF 


SUCCESS 


FROM 


CAUSE 


3 


IN 


PHASE 


3 


88 


PROB. OF 


SUCCESS 


FROM 


CAUSE 


3 


IN 


PHASE 


4 


90 


PROB. OF 


SUCCESS 


FROM 


CAUSE 


3 


IN 


PHASE 


5 


93 


PROB. OF 


SUCCESS 


FROM 


CAUSE 


3 


IN 


PHASE 


6 


96 


PROB. OF 


SUCCESS 


FROM 


CAUSE 


3 


IN 


PHASE 


7 


98 


PROB. OF 


SUCCESS 


FROM 


CAUSE 


3 


IN 


PHASE 


8 


99 


PROB. OF 


SUCCESS 


FROM 


CAUSE 


3 


IN 


PHASE 


9 


998 


PROB. OF 


SUCCESS 


FROM 


CAUSE 


3 


IN 


PHASE 


10 


83 


PROB. OF 


SUCCESS 


FROM 


CAUSE 


4 


IN 


PHASE 


1 


84 


PROB. OF 


SUCCESS 


FROM 


CAUSE 


4 


IN 


PHASE 


2 


85 


PROB. OF 


SUCCESS 


FROM 


CAUSE 


4 


IN 


PHASE 


3 


87 


PROB. OF 


SUCCESS 


FROM 


CAUSE 


4 


IN 


PHASE 


4 


89 


PROB. OF 


SUCCESS 


FROM 


CAUSE 


4 


IN 


PHASE 


5 


92 


PROB. OF 


SUCCESS 


FROM 


CAUSE 


4 


IN 


PHASE 


6 


94 


PROB. OF 


SUCCESS 


FROM 


CAUSE 


4 


IN 


PHASE 


7 


975 


PROB. OF 


SUCCESS 


FROM 


CAUSE 


4 


IN 


PHASE 


8 


99 


PROB. OF 


SUCCESS 


FROM 


CAUSE 


4 


IN 


PHASE 


9 


998 


PROB. OF 


SUCCESS 


FROM 


CAUSE 


4 


IN 


PHASE 


10 


81 


PROB. OF 


SUCCESS 


FROM 


CAUSE 


5 


IN 


PHASE 


1 


83 


PROB . OF 


SUCCESS 


FROM 


CAUSE 


5 


IN 


PHASE 


2 


84 


PROB . OF 


SUCCESS 


FROM 


CAUSE 


5 


IN 


PHASE 


3 


86 


PROB. OF 


SUCCESS 


FROM 


CAUSE 


5 


IN 


PHASE 


4 


89 


PROB. OF 


SUCCESS 


FROM 


CAUSE 


5 


IN 


PHASE 


5 


91 


PROB. OF 


SUCCESS 


FROM 


CAUSE 


5 


IN 


PHASE 


6 


94 


PROB. OF 


SUCCESS 


FROM 


CAUSE 


5 


IN 


PHASE 


7 
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.961 

.99 

.998 

1 



0.0 

624712.0 

.75 

500 

0 

0 

0 

0 

0 

0 

0 



PROB. OF SUCCESS FROM CAUSE 5 IN PHASE 8 

PROB. OF SUCCESS FROM CAUSE 5 IN PHASE 9 

PROB. OF SUCCESS FROM CAUSE 5 IN PHASE 10 

NUMBER OF TRIALS AFTER FAILURE BEFORE A 
DISCOUNT IS APPLIED 

FRACTION EACH FAILURE IS DISCOUNTED 
RANDOM NUMBER SEED FOR GGUBFS UNIFORM (0,1) 
GENERATOR 

FRACTION RELIABILITY IMPROVES AFTER FAILING 
IN A PHASE 

NUMBER OF DESIRED REPETITIONS FOR THE 
SIMULATION 

INTERMEDIATE INPUT OPTION ( 1 : INT . OUTPUT; 

0: NO INT. OUTPUT) 

SAVE ALL MLE W/ DISCOUNTING ESTIMATES 
(1: YES; 0: NO ) 

SAVE ALL MLE SINGLE PHASE ESTIMATES 
(1: YES; 0: NO ) 

SAVE ALL UNWT'D REGRESSION ESTIMATES 
(1; YES; 0: NO ) 

SAVE ALL METHOD 1 WT ' D REG. ESTIMATES 
(1: YES; 0: NO ) 

SAVE ALL METHOD 2 WT ' D REG. ESTIMATES 
(1: YES; 0: NO ) 

SAVE ALL METHOD 3 WT ' D REG. ESTIMATES 
(1: YES; 0: NO ) 
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2: LLOYD 



1 DISCOUNTING OPTION (1: STRAIGHT % ; 

METHOD) 

.9 PERCENT C.I. FOR LLOYD DISCOUNTING METHOD 

( MUST HAVE A VALUE ) 

1 LLOYD DISCOUNT INTERVAL 

.03 WEIGHT FOR PHASE 1 

.03 WEIGHT FOR PHASE 2 

.03 WEIGHT FOR PHASE 1 

.03 WEIGHT FOR PHASE 2 

.03 WEIGHT FOR PHASE 1 

.15 WEIGHT FOR PHASE 2 

.15 WEIGHT FOR PHASE 1 

. 15 WEIGHT FOR PHASE 8 

.2 WEIGHT FOR PHASE 9 

.2 WEIGHT FOR PHASE 10 

3. Sample simulation run, using DISCONT EXEC Al, recorded 
session 

BEGIN RECORDING OF TERMINAL SESSION 
Ready; T=0.01/0.04 08:49:01 
DISCONT 

THIS PROGRAM USES A DISCRETIZED CONTINUOUS MODEL 
DO YOU NEED TO COMPILE YOUR PROGRAM ? (Y/N) 

Y 

VS FORTRAN COMPILER ENTERED. 08:49:17 
+2S=$. 33 
+2S=$. 33 
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**MAIN** END OF COMPILATION 1 ****** 



VS FORTRAN COMPILER EXITED. 08:49:26 

Execution begins. . . 

+2S=$. 33 
+2S=$. 33 
+2S=$. 33 
+2S=$. 33 
+2S=$. 33 

LLOYD ESTIMATION/EXPECTATION OUTPUT IS IN "LLOYD EST Al" 

CONTROL OUTPUT IS IN THE FILE "GROWTH CONTROL Al" 

YOUR ESTIMATION OUTPUT IS IN THE FILE GROWTH EST Al 
DO YOU WISH TO BROWSE YOUR ESTIMATION OUTPUT? (Y/N) 

Y 

PRINT YOUR OUTPUT FILE? (Y/N) 

Y 

DO YOU WISH TO CHANGE THE INPUT DATA FILE (Y/N) 

Y 

DO YOU WISH TO RUN THE PROGRAM AGAIN? (Y/N) 

Y 

VS FORTRAN COMPILER ENTERED. 08:50:56 
+2S=$ .33 
+2S=$. 33 
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**MAIN** END OF COMPILATION 1 ****** 



VS FORTRAN COMPILER EXITED. 08:51:05 



Execution begins... 

+2S=$.33 
+2S=$.33 
+2S=$. 33 
+2S=$. 33 
+2S=$.33 

LLOYD ESTIMATION/EXPECTATION OUTPUT IS IN "LLOYD EST Al" 

CONTROL OUTPUT IS IN THE FILE "GROWTH CONTROL Al" 

YOUR ESTIMATION OUTPUT IS IN THE FILE GROWTH EST Al 
DO YOU WISH TO BROWSE YOUR ESTIMATION OUTPUT? (Y/N) 

Y 

PRINT YOUR OUTPUT FILE? (Y/N) 

N 

DO YOU WISH TO CHANGE THE INPUT DATA FILE (Y/N) 

N 

DO YOU WISH TO RUN THE PROGRAM AGAIN? (Y/N) 

N 

Ready; T=27 . 06/30 . 92 08:52:03 
EXEC REC OFF 



54 



4 . Sample output GROWTH EST A1 



SIMULATION INPUT FILE : PT1C5F3 



PHASE ACTREL 

1 .39842 

2 .42811 

3 .48079 

4 .53924 

5 .60995 

6 .70268 

7 .79812 

8 .89996 

9 .95099 

10 .99004 



LINREL MSEL 
.39805 .0397 
.40747 .0404 
.45315 .0418 
.51145 .0423 
.56290 .0437 
.65305 .0298 
.75894 .0221 
.88128 .0071 
.93208 .0043 
.98662 .0002 



RELEXP HSEE 
.55853 .0375 
.56384 .0304 
.59035 .0250 
.62577 .0217 
.65869 .0177 
.71604 .0122 
.79329 .0103 
.89087 .0045 
.93596 .0027 
.98678 .0001 



MLEWD MSEHD 
.33786 .0517 
.34822 .0547 
.39846 .0564 
.46259 .0561 
.51918 .0585 
.61836 .0402 
.73483 .0289 
.86940 .0091 
.92529 .0054 
.98528 .0002 



REGEST MSEREG 
.36841 .0553 
.37926 .0569 
.43754 .0432 
.50047 .0374 
.56433 .0296 
.64388 .0213 
.73642 .0147 
.84127 .0069 
.90832 .0030 
.96385 .0009 



5. 



Main variables used for the discrete modification of the 
continuous AMSAA model ( AMSAA-D model ) 

TRTOT number of trials in a phase 

NFAPH(K) number of failures in phase k 

NFAPHK number of failures in a phase 

FRATE failure rate estimate of the AMSAA-D model 

FRHAT(K) failure rate estimate for phase k using the 
AMSAA-D model 

FSHAT(K) minimum variance unbiased failure rate 
estimate for phase k 



XAHAT,XBHAT 



Z** 

AREL(K) 

RHAT(K) 

XI 



YI 



regression estimates a, b in one phase and 

repetition 

auxiliary variables 

actual reliability in phase k 

failure rate estimate at phase k 

In of the total number of trials up to phase k 

In of the failure rate estimate at phase k 
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LSSUM(4 ,K) 



LSSUM(4 ,K) 
ANUMER 


A 

summands of the least square estimate a in 
phase k 

A 

numerator to compute a 


ADENOM 


A 

denominator to compute a 


ESTRN(K) 


failure rate estimate in phase k using the 
linear model 


ESTRNE(K) 


failure rate estimate in phase k using the 
exponential power model 


RNHAT(K) 


summation of the reliability estimate using 
the linear model 


RNHATE (K) 


summation of the reliability estimate using 
the exponential power model. 


AVFR(K) 


average failure rate estimate for phase k 


AVRN(K) 


average reliability estimate for phase k, 
using the linear model 


MSE(K) 


Mean Square Error of the linear model 
reliability estimation 


AVRNE(K) 


average reliability estimate for phase k, 
using the exponential power model 


MSEE(K) 


Mean Square Error of the exponential power 
model reliability estimation 


MLEWD(3,K) 


average reliability using the maximum 

likelihood estimation with discounting 


MSEWD(K) 


Mean Square Error of the MLEwd 
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REGEST(3,K) 


average reliability using the exponential 
regression estimate 


MSEREG(K) 


Mean Square Error of the exp. Regr .Estimate 


MSEERR(I,K) 


average Mean Square Error over all phases k 
for the i methods. 


i = 


1 : AMSAA-D, linear model 


i = 


2 : AMSAA-D, exponential power model 


i = 


3 : MLEwd 


i = 


4 : exp. Regr. Estimate 


AVGMSE(I) 


average Mean Square Error over all phases for 
method i 
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APPENDIX C 



FAILURE RATE FOR THE GEOMETRIC AND PASCAL DISTRIBUTION 



The failure rate for a discrete probability distribution 
is defined as 



^ P[N=n] 



1,2,3... 



( C.l ) 



In this treatment the random variable N is the number of 
trials up to and including the r^^ failure; that is, N has a 
Pascal distribution. 

1. Case 1 : The n umb er of failures equals 1. 

In this case the number of Bernoulli trials, N, 
required to obtain the first failure, has a geometric 
distribution with parameter p. Thus : 



P[N = n] = pn-l(i-p) , n = 1,2,... ( C.2 ) 



It follows that 



P[N > n] = p^ 
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Therefore from equation ( C.l ) , the failure rate 



function fj^ is 



pn-1 

or 

f = 1 - p . ( C.3 ) 

The parameter p of the geometric distribution represents the 
reliability of the item under test and therefore it follows 
that 

Rtk = 1 “ f tk ( ^ ) 

This relation suggests the use of equation (II. 6) to convert 
the estimated failure rate into a reliability value; i.e. 

^tk = 1 - ^tk ( C-5 ) 

Even when the number of failures in a phase is r > 1, 
we are still interested in modeling the growth of the 
probability of success on a single trial. Consequently 
equation C.5 provides the correct expression for an estimate 
of reliability in terms of the failure rate estimate. 
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2 . 



Case 2 : The mxmber of failures equals r. 

In this case N has a Pascal distribution with 



parameters p and r. 



P[N = n] = - P)^ 



and 



P[N > n] = P[N > n-1] = P[Y < r-1] 



where Y is binomial with parameters 1-p and n-1. Therefore 



j-w 

It is important to realize that even though C.6 is the 
failure rate function when testing until r failures occur, it 
is not this failure rate that we are modeling in the AMSAA-D 
model. We are modeling the growth of p only; i.e. the growth 
of the probability of success ( reliability ) on a single 
trial . 



P[N > n] = (1 - p)^ 

j=0 ^ 



and 



f 




( C.6 ) 
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APPENDIX D 



FIGURES 16 TO 45 
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RELIABILITY OROWTH ESTIMATION 




e 2 4 6 8 It 

ph«9«a 



Figure 16 : Reliability growth estimation with the discrete 
AMSAA model, pattern 2 
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RELIf^ILITY QPOWTH ESTIMATION 
AMSAA pattern 4 




Figure 17 : Reliability growth estimation with the discrete 
AMSAA model, pattern 4 
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RCLIABIUXTY OROWTM RSTIMATION 




MEAN SqUAPE RPROP 

Pattam I, oaoaaa ■ 3, faiXura/pHaaa b 3 




Figure 18 : Reliability growth estimation and Mean Square 

Error, pattern 1, 3 failure causes, 3 failures/phase 
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RCLIW5ILITY GROWTH ESTIMATION 
Pattern 2, caueee » 3> f aiXure/phae* * 3 
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Figure 19 : Reliability growth estimation and Mean Square 

Error, pattern 2, 3 failure causes, 3 failures/phase 
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RELIABILITY GROWTH ESTIMATION 




MEAN SQUARE ERROR 




Figure 20 : Reliability growth estimation and Mean Square 

Error, pattern 3, 3 failure causes, 3 failures/phase 
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RELIABILITY OROWTH ESTIMATION 
Pattern 4^ oauaae » 3# f ailure/pfiaee ■ 3 
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Figure 21 : Reliability growth estimation and Mean Square 

Error, pattern 4, 3 failure causes, 3 failures/phase 
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RCLIAOILITV OROVTTH ESTiriATIOt4 




MEAN SQUARE ERW5R 
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Figure 22 : Reliability growth estimation and Mean Square 

Error, pattern 5, 3 failure causes, 3 failures/phase 
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Fxgure 23 : Reliability growth estimation and Mean Square 

Error, pattern 6, 3 failure causes, 3 failures/phase 
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Figure 24 : Reliability growth estimation and Mean Square 

Error, pattern 1, 5 failure causes, 1 failure/phase 
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RELIABILITY GROWTH ESTIHATIOM 
Pattern C> oacr««« • 6, failur«/pHM» « 1 




MEAN SQUARE ERROR 




pMjmaa 

Figure 25 : Reliability growth estimation and Mean Square 

Error, pattern 2, 5 failure causes, 1 failure/phase 
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PCLIABILITY OmjWTH ESTIMATION 
Pattern 3^ oaLieaa ■ 6^ f ailura/pMaea ■ 1 




MEAN SQUARE ERROR 




pKaaaa 

Figure 26 : Reliability growth estimation and Mean Square 

Error, pattern 3, 5 failure causes, 1 failure/phase 
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MEAN SQUARE ERROR 
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Figure 27 : Rel iahi 1 j ty growth estimation and Mean Square 

Error, pattern 4, 5 failure causes, 1 failure/phase 



73 



RELIABILITY OROWTH ESTIMATION 




MEAN SQUARE ERROR 







Figure 28 : Reliability growth estimation and Mean Square 

Error, pattern 5, 5 failure causes, 1 failure/phase 
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Figure 29 : Reliability growth estimation and Mean Square 

Error, pattern 6, 5 failure causes, 1 failure/phase 
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Figure 30 : Reliability growth estimation and Mean Square 

Error, pattern 1, 7 failure causes, 3 failures/phase 
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Figure 31 : Reliability growth estimation and Mean Square 

Error, pattern 2, 7 failure causes, 3 failures/phase 
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Figure 32 : Reliability growth estimation and Mean Square 

Error, pattern 3, 7 failure causes, 3 failures/phase 



78 



RELIABILITY GROWTH ESTI«ATIOt< 
P«tt(»m 4, amummm ■ 7, fmiiurm/'piAmnm ■ 3 




MEAN SQUARE ERROR 







Figure 33 : Reliability growth estimation and Mean Square 

Error, pattern 4, 7 failure causes, 3 failures/phase 
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Figure 34 : Reliability growth estimation and Mean Square 

Error, pattern 5, 7 failure causes, 3 failures/phase 
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Figure 35 : Reliability growth estimation and Mean Square 

Error, pattern 6, 7 failure causes, 3 failures/phase 
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failures/phase = 3 
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Figure 37 : Mean Square Error for both AMSAA-D models as a 
function of phases and number of failure causes, pattern 3, 
failures/phase = 3 
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Figure 38 : Mean Square Error for both AMSAA-D models as a 
function of phases and number of failure causes, pattern 4, 
failures/phase = 3 
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Figure 39 : Mean Square Error for both AMSAA-D models as a 
function of phases and number of failure causes, pattern 5, 
failures/phase = 3 
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Figure 42 : Mean Square Error for both AMSAA-D models as a 
function of phases and f ailures/phase , pattern 3, failure 
causes = 5 
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Figure 43 : Mean Square Error for both AMSAA-D models as a 
function of phases and failures/phase, pattern 4, failure 
causes = 5 



89 




causes = 5 



90 




Figure 45 : Mean Square Error for both AMSAA-D models as a 
function of phases and failures/phase, pattern 6, failure 
causes = 5 
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APPENDIX E 



GLOSSARY OF THE DIFFERENT RELIABILITY GROWTH MODELS 

1. Maximum Likelihood Estimate Model 

2. The Exponential Regression Model 

3 . The Discrete AMSAA Model 

4 . The AMSAA-D Model 

a. Linear AMSAA-D Model 

b. Exponential Power AMSAA-D Model 
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1 . 



Maximum Likelihood Estimate Model 



A 

The reliability estimate 



of the true reliability R is 



computed as 




total trials - total failures 
total trials 



If failure discounting is applied then the reliability 



A 

estimate R^^, 



is evaluated as 



A 



R 



tk 



total trials - total adjusted failures 
total trials 



The 


estimation 


is performed 


after each 


phase 


and, if 


failure 


discounting 


is applied, 


the actual 


test 


data are 



updated with the use of test data from previous phases. This 
model is denoted as MLEwd. 

2. The Exponential Regression Model 

The exponential regression model obtains sequentially 

A 

updated estimates R^^^ of the true reliability R^j^ after the 
t h 

k phase. The basic model for R., is : 



R 



tk 



1 




The parameter 



A 

= a + /Sk is estimated by A, where 



A 




^k + 



93 



The estimates and are least square regression estimates 

A A 

for a and Equations for and are developed in 

Reference 4 . 

A 

The exponential regression estimate of the true 

reliability R^j^ is then finally : 



^tk = 1 - e ^k^^ , k = 1,2,... 



This model is denoted as exp. Regr. Estimate. 

3 - The Discrete AMSAA Model 

In the Discrete AMSAA model, the equation for the 

t h. 

probability of success, Rj^, on each trial in the i^ testing 
phase is given by 



(1 - 






= - 



XT 



k-1 



where 



Nj, : number of tests in test phase i 



■k 



k 

= 2 
j=l 



N. 



After each new test phase is completed the maximum likelihood 
estimates for X and ^ are recomputed to provide the MLE for 
Rj^. The two equations which must be solved after testing is 
completed in phase k are as follows [Ref. 6]: 
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E 



/ M, 


Ni - Mi \ 


AT^lriT— AT^ InT i 


\ [ATf-ATf_; 




Ni - ATf + ATf_, 







= 0 



and, 



E 



[ 


N. - M. [ 




\ [ATf-ATf_,' 




N,- - ATf + ATf_, 







= 0 



Numerical methods and a computer are needed to solve these 
equations . 

4. The AMSAA-D Model 

The Continuous AMSAA Model expresses the instantaneous 
failure rate after TT. time units of testing by 

TTr ^ 

= b(l - a)TTj^^ 

where TT^, is the total accumulated test time over k phases. 
This model is also known as the Duane Model. After examining 
a large variety of data, Duane noticed that the plot of the 
cumulative failure rate versus cumulative test time on log- 
log paper produced a straight line. From this he inferred a 
linear relationship between log and log T. Consequently, 
he wrote the following model for cumulative failure rate 

A^ = kT"^ 
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where 






T 



= N(T) 
T 



Thus 



N(T) : total failures up to time T 
k and a are unknown parameters . 



N(T) 

T 

N(T) 



kT 



kT^ 



-a 



The instantaneous failure rate f^ is the change in N(T) per 
unit time. Thus 



dN(T) 
T dT 



(1 - a)kT 



The AMSAA model departs at this point from Duane by using 
maximum likelihood methods to estimate the unknown parameters. 
Letting a = 1 -^ we have 

f^ = ^kT^“^ 

which is the Weibull failure rate function. Consequently AMSAA 
develops maximum likelihood estimates for and k assuming 
time to failure within a phase has a Weibull distribution. 
This assumption makes the accuracy of their model succeptible 
to variations in the underlying failure distribution. Woods 
[Ref. 4] uses regression methods to estimate the parameters 
and does not assume an underlying distribution. This makes his 
procedure more robust. Similar procedures are followed in the 
AMSAA-D model. 
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The AMSAA-D model relates the failure rate and the end 
of k test phases to the total number of failures across 
the first k test phases by 



^tk - t»(l - 



a. Linear AMSAA-D Model 

A 

In this model the reliability estimate of the true 
reliability at the end of phase k is computed as 



R 



tk 



1 



f 



tk‘ 



b. The exponential Power AMSAA-D Model 

A 

In this model the reliability estimate R^j^ of the true 
reliability R^j^ at the end of phase k is computed as 



A 
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